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Abstract 



In this paper, we quantify the statistical coherence between financial time series by means of 
the Renyi entropy. With the help of Campbell's coding theorem we show that the Renyi entropy 
selectively emphasizes only certain sectors of the underlying empirical distribution while strongly 
suppressing others. This accentuation is controlled with Renyi's parameter q. To tackle the issue 
of the information flow between time series we formulate the concept of Renyi's transfer entropy as 
a measure of information that is transferred only between certain parts of underlying distributions. 
This is particularly pertinent in financial time series where the knowledge of marginal events such 
as spikes or sudden jumps is of a crucial importance. We apply the Renyian information flow to 
stock market time series from 11 world stock indices as sampled at a daily rate in the time pe- 
riod 02.01.1990 - 31.12.2009. Corresponding heat maps and net information flows are represented 
graphically. A detailed discussion of the transfer entropy between the DAX and S&P500 indices 
based on minute tick data gathered in the period from 02.04.2008 to 11.09.2009 is also provided. 
Our analysis shows that the bivariate information flow between world markets is strongly asym- 
metric with a distinct information surplus flowing from the Asia-Pacific region to both European 
and US markets. An important yet less dramatic excess of information also flows from Europe to 
the US. This is particularly clearly seen from a careful analysis of Renyi information flow between 
the DAX and S&P500 indices. 
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I. INTRODUCTION 



The evolution of many complex systems in natural, economical, and social sciences is 
usually presented in the form of time series. In order to analyze time series, several of 
statistical measures have been introduced in the literature. These include such concepts as 
probability distributions [H, 0], autocorrelations j^J, mult i- fractals [if, complexity 0, 5[, or 
entropy densities j^]. Recently, it has been pointed out that the transfer entropy (TE) is a 
very useful instrument in quantifying statistical coherence between time evolving statistical 
systems In particular, in Schreiber's paper j(| it was demonstrated that TE is espe- 

cially expedient when global properties of time series are analyzed. Prominent applications 
are in multivariate analysis of time series, including e.g., study of multichannel physiological 
data or bivariate analysis of historical stock exchange indices. Methods based on TE have 
substantial computational advantages which are particularly important in analyzing a large 
amount of data. In all past works, including , [loj], the emphasis has been on various 
generalizations of transfer entropies that were firmly rooted in the framework of Shannon's 
information theory. These, so called Shannonian transfer entropies are, indeed, natural can- 
didates due to their ability to quantify in a non-parametric and in explicitly non-symmetric 
way the flow of information between two time series. An ideal testing ground for various TE 
concepts are financial-market time series because of the immense amount of electronically 
recorded financial data. 

Recently, economy has become an active research area for physicists. They have in- 
vestigated stock markets using statistical-physics methods, such as the percolation theory, 
multifractals, spin-glass models, information theory, complex networks, path integrals, etc.. 
In this connection the name econophysics has been coined to denote this new hybrid field on 
the border between statistical physic and (quantitative) finance. In the framework of econo- 
physics it has became steadily evident that the market interactions are highly nonlinear, 
unstable, and long-ranged. It has also became apparent that all agents (e.g., companies) 
involved in a given stock market exhibit interconnectedness and correlations which represent 
important internal force of the market. Typically one uses correlation functions to study 
the internal cross-correlations between various market activities. The correlation functions 
have, however, at least two limitations: First, they measure only linear relations, although 
it is clear that linear models do not faithfully reflect real market interactions. Second, 
all they determine is whether two time series (e.g., two stock-index series) have correlated 
movement. They, however, do not indicate which series affects which, or in other words, 
they do not provide any directional information about cause and effect. Some authors use 
such concepts as time-delayed correlation or time-delayed mutual information in order to 
construct asymmetric "correlation" matrices with inherent directionality. This procedure is 
in many respects ad hoc as it does not provide any natural measure (or quantifier) of the 
information flow between involved series. 

In the present paper we study multivariate properties of stock-index time with the help 
of econophysics paradigm. In order to quantify the information flow between two or more 
stock indices we generalize Schreibers' Shannonian transfer entropy to Renyi's information 
setting. With this we demonstrate that the corresponding new transfer entropy provides 
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more detailed information concerning the excess (or lack) of information in various parts of 
the underlying distribution resulting from updating the distribution on the condition that 
a second time series is known. This is particularly relevant in the context of financial time 
series where the knowledge of tale-part (or marginal) events such as spikes or sudden jumps 
bears direct implications, e.g., in various risk-reducing formulas in portfolio theory. 

The paper is organized as follows: In Section |TT] we provide some information-theoretic 
background on Shannon and Renyi entropies (RE's). In particular, we identify the condi- 



tional Renyi's entropy with the information measure introduced in Ref. 11]. Apart from 



satisfying the chain rule (i.e., rule of additivity of information) the latter has many desirable 
properties that are to be expected from a conditional information measure. Another key con- 
cept — the mutual Renyi entropy, is then introduced in a close analogy with Shannon's case. 
The ensuing properties are also discussed. Shannonian transfer entropy of Schreiber is briefly 
reviewed in Section III II There we also comment on effective transfer entropy of Marchinski 
et all. The core quantity of this work — the Renyian transfer entropy (RTE), is motivated 
and derived in Section IIVI In contrast to Shannonian case, the Renyian transfer entropy is 
generally not positive semi-definite. This is because RE non-linearly emphasizes different 
parts of involved probability density functions (PDF's). With the help of Campbell's coding 
theorem we show that the RTE rates a gain/loss in risk involved in a next-time-step behav- 
ior in a given stochastic process, say X, resulting from learning a new information, namely 
historical behavior of another (generally cross-correlated) process, say Y. In this view the 
RTE can serve as a convenient rating factor of a riskiness in inter-connected markets. We 
also show that Renyian transfer entropy allows to amend spurious effects caused by a finite 
size of a real data set which in Shannon's context must be, otherwise, solved by means 
of the surrogate data technique and ensuing effective transfer entropy. In Section [V] we 
demonstrate the usefulness and formal consistency of RTE by analyzing cross-correlations 
in various international stock markets. On a qualitative level we use 183.308 simultaneously 
recorded data points of the eleven stock exchange indices, sampled at a daily (end-of-trading 
day) rate to construct the heat maps and net flows for both Shannon's and Renyi's infor- 
mation flows. On a quantitative level we explicitly discuss time series from the DAX and 
S&P500 market indices gathered on a minute-tick basis in the period from December 1990 
till November 2009 in the German stock exchange market (Deutsche Borse). Presented 
calculations of Renyi and Shannon transfer entropies are based on symbolic coding compu- 
tation with the open source software R. Our numerical results imply that RTE's among 
world markets are typically very asymmetric. For instance, we show that there is a strong 
surplus of an information flow from the Asia- Pacific region to both Europe and the U.S. A 
surplus of the information flow can be also observed to exists from Europe to the U.S. In 
this last case the substantial volume of transferred information comes from tail-part (i.e., 
risky part) of underlying asset distributions. So, despite the fact that the U.S. contributes 
more than half of the world trading volume, this is not so with information flow. 

Further salient issues, such as dependence of RTE on Renyi's q parameter or on the 
data block length are numerically also investigated. In this connection we find that the 
cross-correlation between DAX and S&P500 has a long-time memory which is around 200- 
300 mins. This should be contrasted with typical memory of stock returns which are of 
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the order of seconds or maximally few minutes. Various remarks and generalizations are 
proposed in the concluding Section I VI I For reader's convenience we give in Appendix A a 
brief dictionary of market Indices used in the main text and in Appendix B we tabulate an 
explicit values of effective transfer entropies used in the construction of heat maps and net 
information flows. 

II. INFORMATION-THEORETIC ENTROPIES OF SHANNON AND RENYI 

In order to express numerically an amount of information that is shared or transferred 
between various data sets (e.g., two or more random processes), one commonly resorts to 
information theory and especially to the concept of entropy. In this section we briefly review 
some essentials of Shannon's and Renyi's entropy that will be needed in following sections. 



A. Shannon's entropy 



The entropy concept was originally introduced by Clausius 12] in the framework of ther- 
modynamics. By analyzing a Carnot engine he was able to identify a new state function 
which never decreases in isolated systems. The microphysical origin of Clausius' phenomeno- 
logical entropy was clarified more than 20 years later in works of Boltzman and (yet later) 
Gibbs who associated Clausius entropy with the number of allowed microscopic states com- 
patible with a given observed macrostate. The ensuing Boltzmann- Gibbs entropy reads 



w 



h B g{v) = -k B 22 ln K x ) 



(1) 



x e x 



where ks is Boltzmann's constant, X is the set of all accessible microstates compatible with 
whatever macroscopic observable (state variable) one controls and W denotes the number 
of microstates. 

It should be said that the passage from Boltzmann-Gibbs to Clausius entropy is estab- 
lished only when the conditional extremum V cx of Hbg subject to the constraints imposed 
by observed state variables is inserted back into Hbg- Only when this maximal entropy 
prescription [l3| is utilized H B g turns out to be a thermodynamic state function and not 
mere functional on a probability space. 

In information theory, on the other hand, the interest was in an optimal coding of a 
given source data. By optimal code is meant the shortest averaged code from which one can 
uniquely decode the source data. Optimality of coding was solved by Shannon in his 1948 
seminal paper 14J. According to Shannon's source coding theorem 14j, [15], the quantity 



w 



(2) 



x ex 



corresponds to the averaged number of bits needed to optimally encode (or "zip" ) the source 
dataset X with the source probability distribution V(X). On a quantitative level (T5]) rep- 
resents (in bits) the minimal number of binary (yes/no) questions that brings us from our 
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present state of knowledge about the system X to the one of certainty [14|,ll6|,|l7l. It should 
be stressed that in Shannon's formulation X represents a discrete set (e.g., processes with 
discrete time), and this will be also the case here. Apart from the foregoing operational 
definitions, Eq. ([2]) has also several axiomatic underpinnings. Axiomatic approaches were 
advanced by Shannon . [l5| . Khinchin Fadeev [jji] an others {2o|. The quantity (j2J) 
has became known as Shannon's entropy (SE). 

There is an intimate connection between Boltzmann-Gibbs entropy and Shannon's en- 
tropy. In fact, thermodynamics can be viewed as a specific application of Shannon's in- 
formation theory: the thermodynamic entropy may be interpreted (when rescaled to "bit" 
units) as the amount of Shannon information needed to define the detailed microscopic state 
of the system, that remains "uncommunicated" by a description that is solely in terms of 



thermodynamic state variables 2lN23 



Among important properties of SE is its concavity in V, i.e. for any pair of distributions 
V and Q, and a real number < A < 1 holds 

H(XV + (1-X)Q) > XH{V) + (1 - X)H(Q) . (3) 

Eq. ()3]) follows from Jensen's inequality and a convexity of xlogx for x > 0. Concavity 
is an important concept since it ensures that any maximizer found by the methods of the 
differential calculus yields an absolute maximum rather than a relative maximum or mini- 
mum or saddle point. At the same time it is just a sufficient (i.e., not necessary) condition 
guarantying a unique maximizer. It is often customary to denote SE of the source X as 
H(X) rather than H(V). Note that SE is generally not convex in X\ 

It should be stressed that the entropy fl2]) really represents a self-information: the infor- 
mation yielded by a random process about itself. A step further from a self-information 
offers the joint entropy of two random variables X and Y which is defined as 

H(X fl Y) = - p(x,y)\og 2 p(x 1 y) 1 (4) 

x G X, y £ Y 

and which represents the amount of information gained by observing jointly two (generally 
dependent or correlated) statistical events. 

A further concept that will be needed here is the conditional entropy of X given Y, which 
can be motivated as follows: Let us have two statistical events X and Y and let event Y 
has a sharp value y, then the gain of information obtained by observing X is 

H(X\Y = y) = ^ p{x\y)\og 2 p{x\y) . (5) 

x 6 X 

Here the conditional probability p{x\y) = p{x,y)/p(y). For general random Y one defines 
the conditional entropy as the averaged Shannon entropy yielded by X under the assumption 
that the value of Y is known, i.e. 

h(x\y) = j2p(yW x \ Y = y) = - E p{^v>g 2 p{Av) ■ (6) 

y €Y x £ X,y €Y 
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From ([6]), in particular, follows that 

H(X fl Y) = H(Y) + H(X\Y) = H(X) + H(Y\X) . (7) 

Identity ([7]) is known as additivity (or chain) rule for Shannon's entropy. In statistical ther- 
modynamics this rule allows to explain, e.g., Gibbs paradox. Applying Eq. ([7j) iteratively, 
we obtain: 

H{x 1 n x 2 n ■ ■ ■ n x n ) = H(x x ) + h(x 2 \x 1 ) + H(x 3 \x 1 nx 2 ) + ■■• 

n 

= J]iy(x i |x 1 nx 2 n---nx i _ 1 ). (8) 

i 

Another relevant quantity that will be needed is the mutual information between X and 
Y. This is defined as: 

I(X;Y) = £ p(x , y)log2 ^jL (9) 

and can be equivalently written as 

I(X;Y) = H(X) - H{X\Y) = H(Y) - H(Y\X) . (10) 

This shows that the mutual information measures the average reduction in uncertainty (i.e., 
gain in information) about X resulting from observation of Y. Of course, the amount of 
information contained in X about itself is just the Shannon entropy: 

I(X;X) = H(X). (11) 

Notice also that from Eq. fl9]) follows I(X; Y) = I(Y; X) and so X provides the same 
amount of information on Y as Y does on X. For this reasons the mutual information is not 
a useful measure to quantify a flow of information. In fact, the flow of information should 
be by its very definition directional. 

In the following we will also find useful the concept of conditional mutual entropy between 
X and Y given Z which is defined as 

I{X;Y\Z) = H{X\Z) - H(X\Yf]Z), 

= I(X;YnZ) - I(X;Y). (12) 

The latter quantifies the averaged mutual information between X and Y provided that Z is 
known. Applying ( IT2l) and ( jTOl iteratively we may write 

/(x ; y 1 n---ny n |z 1 n---nz m ) = e{x\z x n •••nzj 

- H(x\Y 1 n---nY n nz 1 n---nz m ) 

= i(X; Y 1 n • • • n Y n n z x n • • • n z m ) 

- i(x ; z 1 n---nz m ). (13) 

For further details on the basic concepts of Shannon's information theory, we refer the 



reader to classical books, e.g., Ash [17J and, more recently, Csiszar and Shields 



241. 
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B. Renyi's entropy 



Renyi introduced in Refs. [25|, |26 



a one-parameter family of information measures 
presently known as Renyi entropies [ll|, |25j. In practice, however, only a singular name 
- Renyi's entropy — is used. RE of order q (q > 0) of a distribution V on a finite set X is 
defined as 



S <f) {V) 



1 



1-q 



log 2 p< 



x 



(14) 



For RE (1141) one can also formulate source coding theorem. While in the Shannon case the 
cost of a code-word is a linear function of the length — so the optimal code has a minimal 
cost out of all codes, in the Renyi case the cost of a code-word is an exponential function 



of its length [27H29 



This is, in a nutshell, an essence of the so-called Campbell's coding 
theorem (CCT). According to this RE corresponds to the averaged number of bits needed 
to optimally encode the discrete source X with the probability V(X), provided that the 



codeword-lengths are exponentially weighted [30[. From the form ( TT4~1) one can easily see 
that for q > 1 RE depends more on the probabilities of the more probable values and less 
on the improbable ones. This dependence is more pronounced for higher q. On the other 
hand, for < q < 1 marginal events are accentuated with decreasing q. In this connection 
we should also point out that Campbell's coding theorem for RE is equivalent to Shannon's 



coding theorem for SE provided one uses instead of p(x) the escort distribution [29 



Q q [x) 



p q (x) 



(15) 



The PDF Q q (x) was first introduced by Renyi (26| and in the physical context brought by 



Beck, Schlogl, Kadanoff and others (see, e.g., Refs. |3jJ, |32j). Note (cf. Fig. CP that for 



P 



0. 6 




FIG. 1: The function p q for event probability p and varying Renyi's parameter q. Arrows indicate 
decreasing values of q for < q < 1 (dark arrow) or increasing values of q for q > 1 (lighter arrow). 



q > 1 the escort distribution emphasizes the more probable events and de-emphasizes more 
improbable ones. This trend is more pronounced for higher values of q. For < q < 1 
the escort distribution accentuates more improbable (i.e., marginal or rare) events. This 



S 



FIG. 2: A plot of the escort distribution for two-dimensional V: Q q = p q /(p q + (1 — p) q ). 



dependence is more pronounced for decreasing q. This fact is clearly seen on Fig. |5J So by 
choosing different q we can "scan" or "probe" different parts of the involved PDF's. 

It should be stressed that apart from CCT, RE has yet further operational definitions, 
e.g., in the theory of guessing 33], in the buffer overflow problem 34|or in the theory of error 
block coding 35[. RE is also underpinned with various axiomatics 25|, |26|, |36| . In particular, 
it satisfies identical Khinchin axioms 18| as Shannon's entropy save for the additivity axiom 
(chain rule) [ll|, H : 



sj. R \xnY) = S {R \Y) + S {R) (X\Y) 



(16) 



where the conditional entropy Sq (X\Y) is defined with the help of the escort distribution 
( |T5l) (see, e.g., Refs. Ill, |3jJ, |39j). For q — > 1 RE reduces to the Shannon entropy: 



limSi^ = H. 



(17) 



as one can easily verify with rHospital's rule. 

We define the joint Renyi entropy (or the joint entropy of order q) for two random 
variables X and Y in a natural way as: 

S q R) (XHY) = 



<1 



iog 2 p q ( x >y"> 



(18) 



x e x 



The conditional entropy of order q of X given Y is similarly as in the Shannon case defined 
as the averaged Renyi's entropy yielded by X under the assumption that the value of Y is 



r-t 



known. As shown in Refs. [Ill, |38|, |40| this has the form 



SW(X\Y) 



1-q 
1 



log; 



E 



x G X, y 6 Y 



p q {x\y)q q (y) 



log : 



E y eYi q (y) 

J2xeX,y & YP 9 ( X ^y) 



E y eY<i q (y) 



(19) 
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In this connection it should be mentioned that several alternative definitions of the con- 
ditional RE exist (see, e.g., Refs. [26|, |35|, |4l|), but the formulation (Tl9|) differs from other 
versions in a few important ways that will show up to be desirable in the following consid- 
erations. The conditional entropy defined in (Tl9|) has the following important properties, 
namely [ll|, 4ol 

( R) 

- < SP{X\Y) < log 2 n, where n is a number of elements in X, 

— Sq (X\Y) = only when Y uniquely determines X (i.e., no gain in information), 

- ]hn q ^ 1 S? t \X\Y) = H{X\Y), 

( R) ( R) 

— when X and Y are independent then SP(X\Y) = S v q '(X) . 

Unlike Shannon's case one cannot, however, deduce that the equality S q (X\Y) = 
Sq R \x) implies independency between event X and Y. Also the inequality S q R \X\Y) < 

( Ft} 

(i.e., an extra knowledge about Y lessens our ignorance about X) does not hold 



here in general ll|, |26( . The latter two properties may seem as a serious flaw. We will now 



argue that this is not the case and, in fact, it is even desirable. 

First, in order to understand why S q (X\Y) = Sq (X) does not imply independency 
between X and Y we define the information-distribution function 



■F-p(x) 



■^og 2 p(z)<x 



(20) 



which represents the total probability caused by events with information content H(z) 
— \og 2 p(z) < x. With this we have 



2 ( - 1 -^ x dT v {x) = Yl 2 {1 - q)H{z) p(z) = Yl 

x<H(z)<x+dx x<H(z)<x+dx 



P q (z) 



(21) 



and thus 



Sj, R \X) 



q 



log 2 



2 { - 1 ~ q)x dF- 



(22) 



Taking the inverse Laplace transform with the help of the so-called Post's inversion 



formula [42] we obtain 



T v {x) 



lim 

k— >oo 



fc! 



k 



x In 2 



fc+i 



dq* 



2 (l-<7)sf ) (X) 



(5 



(23) 



k/(x ln2)+l 



Analogous relation holds also for J i 'v\q{. x ) an d associated Sq (X\Y). As a result we see that 

( R\ 

when working with S\ of different orders we receive much more information on underlying 
distribution than when we restrict our investigation to only one q (e.g., to only Shannon's 
entropy). In addition, Eq. (1231) indicates that we need all g > 1 (or equivalently all < q < 1, 



sec 



43|) in order to uniquely identify the underlying PDF. 
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In view of Eq. f l23|) we see that the equality between Sg (X\Y) and Sq (X) at some 
neighborhood of q merely implies that J i 'v\q{ x ) — F-p{x) for some x. This naturally does 
not ensure independency between X and Y. We need equality S q (X\Y) = S q (X) for all 
q > 1 (or for all < q < 1) in order to secure that J--p\q(x) = F-p{x) holds for all x which 
would in turn guarantee that V(X) = V(X\Y). Therefore, all RE with q > 1 (or all with 

(Ft) (Ft) 

< q < 1) are generally required to deduce from S q (X\Y) = S q (X) an independency 
between X and Y. 

In order to understand the meaning; of the inequality S { q R) (X\Y) < S { q R) (X) we first 
introduce the concept of mutual information. The mutual information of order q between 
X and Y can be defined as (cf. Eq. ffTOl) ) 

l\ R \X-Y) = SW(X) - S^(X\Y) 

= SW(X) + S { q R) (Y) - S q R \XnY), (24) 

which explicitly reads 

I^HX-Y) = 1 h Z*ex,yeY<l 9 (v)P q ix) 

q ' l ~i J2 x& x, y eYP q ( x ^y) 

= j_ } E x ex,yeYi q (y)p q (*) 
1-9 og2 E x ex,yeYi q (y)p q (*\y)' 

Note that we have again the symmetry relation I q R \X;Y) = I q (Y;X) as well as the 
consistency condition I q R \X; X) = S q (X). So similarly as in the Shannon case, Renyi's 
mutual information formally quantifies the average reduction in uncertainty (i.e., gain in 
information) about X that results from learning the value of Y, or vice versa. 

From Eq. (\2M we see that the inequality in question, i.e., S q R \x\Y) < S q (X) implies 
Iq R \Y\ X) > 0. According to (1251) this can be violated only when 



x e x x e x 

^2p\x) <^(V q (x\Y)) q for 0<g<l. (26) 

x e x x e x 

Here (. . .) q is an average with respect to the escort distribution g q (y) (see Eq. ( fT5l) ). 

By taking into account properties of the escort distribution, we can deduce that 

(Ft) 

I q (X; Y) < when a larger probability events of X receive by learning Y a lower value. As 
for the marginal events of X, these are by learning Y indeed enhanced, but the enhancement 
rate is smaller than the suppression rate of large probabilities. For instance, this happens 
when 

w = {'- £ -^" ^h ™ = { i y i - i Y £ -^2- -'^2}' (27 > 

for 
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1 



(28) 



< e < 1, 



n > 2. 



The inequality ( )28|) ensures that I(Y; X) > holds. The left inequality in ( )28|) saturates 
when J(F; X) = 0, see also Fig. El 

For < q < 1 is the situation analogous. Here properties of the escort distribution imply 

( Ft) 

that r q n '(Y;X) < when marginal events of X get by learning Y a higher probability. 
The suppression rate for large (i.e. close-to-peak) probabilities is now smaller than the 
enhancement rate of marginal events. This happens, for example, for distributions, 



with e fulfilling again the inequality (128]) . This can be also directly seen from Fig. [3] when 
we revert the sign of l\ R) {Y]X). When we set q — 1 then both inequalities (1261) are simul- 
taneously satisfied yielding I(Y ; X) = — as it should. 

In contrast to a Shannonian case where the mutual information quantifies the average re- 
duction in uncertainty resulting from observing/learning a further information, in the Renyi 
case we should use Campbell's coding theorem in order to properly understand the meaning 
of Iq R \Y; X). According to the CCT Sq (X) corresponds to the minimal average cost of a 



FIG. 3: Example of a typical situation when I q (X;Y) is negative. Distributions V(X) and 
V(X\Y) are specified in ([2T]) and e is chosen so that corresponding I(X;Y) = 0. 

coded message with a non-linear (exponential) weighting/ pricing of codeword- lengths. While 
according to Shannon we never increase ignorance by learning Y (i.e., possible correlations 
between X and Y can only reduce the entropy), in Renyi's setting extra knowledge about 
Y might easily increase the minimal price of coding X because of the nonlinear pricing. 
Since the CCT penalizes long codewords which in Shannon's coding have low probability, 
the price of the X\Y code may easily increase, as we have seen in examples (1271) and (1291) . 




2.1) 
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In the key context of financial time series, the risk valuation of large changes such as 
spikes or sudden jumps is of a crucial importance, e.g., in various risk-reducing formulas in 
portfolio theory. The role of Campbell's pricing can in these cases be interpreted as a risk- 
rating method which puts an exponential premium on rare (i.e., risky) asset fluctuations. 
From this point of view the mutual information Iq (X; Y) represents a rating factor which 
rates a gain/loss in risk in X resulting from learning a new information, namely information 
about Y. 

The conditional mutual information of order q between X and Y given Z is defined as 

l\ R \X-Y\Z) = S<f)(X\Z) - S { q R \X\YnZ). (30) 

Note that because of a validity of the chain rule (fT6l) . relations ([8]) and ffT3|) also hold true 
for the RE. 

To close this section, we shall stress that information entropies are primarily important 
because there are various coding theorems which endow them with an operational (that is, 
experimental) meaning, and not because of intuitively pleasing aspects of their definitions. 
While coding theorems do exist both for the Shannon entropy and the Renyi entropy there 
are (as yet) no such theorems for Tsallis', Kaniadakis', Naudts' and other currently popular 
entropies. The information-theoretic significance of such entropies is thus not obvious. Since 
the information-theoretic aspect of entropies is of a crucial importance here, we will in the 
following focus only on the SE and the RE. 

III. FUNDAMENTALS OF SHANNONIAN TRANSFER ENTROPY 
A. Shannonian transfer entropy 

As seen in Section III Al the mutual information I(X; Y) quantifies the decrease of un- 
certainty about X caused by the knowledge of Y. One could be thus tempted to use it 
as a measure of an informational transfer in general complex systems. A major problem, 
however, is that Shannon's mutual information contains no inherent directionality since 
I(X;Y) = I(Y;X). Some early attempts tried to resolve this complication by artificially 
introducing the directionality via time-lagged random variables. In this way one may define, 
for instance, the time-lagged mutual (or directed Kullback-Leibler) information as 

I(X;Y) t>T = ^ P ( gt ,^ r )bfa gfc^4 . (31) 
^ P{xt)q{yt) 

The later describes the average gain of information when replacing the product probability 
V t xQt = {p(x t )q(yt); %t e X t ,y t G Y t } by the joint probability V t nQ t - T = {p(x t ,yt- T ); %t e 
X t ,yt- T £ Yt- T }. So the information gained is due to cross-correlation effect between ran- 
dom variables X t and Y t (respectively, Y t - T ). It was, however, pointed out in Ref. jf| that 
prescriptions such as fl3T|) . though directional, also take into account some part of the in- 
formation that is statically shared between the two random processes X and Y. In other 
words, these prescriptions do not produce statistical dependences that truly originate only 
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in the stochastic random process Y, but they do include the effects of a common history 
(such as, for example, in the case of a common external driving force). 

For this reason, Schreiber introduced in Ref. the eoneept of (Shannonian) transfer 
entropy (STE). The latter, apart from directionality, accounts only for the cross-correlations 
between statistical time series X and Y whose genuine origin is in the "source" process Y. 
The essence of the approach is the following. Let us have two time sequences described by 
stochastic random variables X t and Y t . Let us assume further that the time steps (data 
ticks) are discrete with the size of an elementary time lag r and with t n = to + nr (to is a 
reference time). 

The transfer entropy Ty^ximJ) can then be defined as 

T y ^ x (m, /) = H(x tm+1 \x tl n • • • n x tm ) - H{x tm+1 \x tl n • • • n x tm n Y t _ l+1 n-nyj 
= i(x tm+1 - x tl n • • • n x tm n Y tm _ l+1 n-nyj- i{x tm+1 - x tl n • • • n x t J . 

(32) 

The last line of (|32|) indicates that Ty^. x ( m J) represents the following. 

+ gain of information about Xt m+1 caused by the whole history of X and Y up to time t m 

— gain of information about Xt m+1 caused by the whole history of X up to time t m 

= gain of information about Xt m+1 caused purely by the whole history of Y up to time t m . 

Note that one may equivalently rewrite ( I3"2"j) as the conditional mutual information 

T Y ^ x (m,i) = i(x tm+1 ;Y tm _ l+1 n---nY tm \x tl n---nx tm ). (33) 

This shows once more the essence of Schreiber's transfer entropy, namely, that it describes 
the gain in information about X tm+1 caused by the whole history of Y (up to time t m ) under 
the assumption that the whole history of X (up to time t m ) is known. According to the 
definition of the conditional mutual information, we can explicitly rewrite Eq. ( 1331) as 

Ty^. x (m,l) 

p(x tl , x tm+1 ,y tm _ l+1 , ...,y t J log 2 — — , (34) 

P\. X t m + l Ftl) • • ■ ) X tm) 

where x t and y t represent the discrete states at time t of X and Y, respectively. 

In passing, we may observe from the first line of f)32p that Ty^x > (any extra knowl- 
edge in conditional entropy lessens the ignorance). In addition, due to the Shannon-Gibbs 
inequality (see, e.g., Ref. [23|), Ty^x = only when 

m + 1 \ X h ' - - - ' X tm ' 2/tm-i + l ' " " " ' Vtm) ^ (35) 

p{x t 

m +l \ X tl ; • • • ; %t 

This, however, means that the history of Y up to time t m has no influence on the value 
of X f ... or, in other words, there is no information flow from Y to X\ i.e., the Y and 
X time series are independent processes. If there is any kind of information flow, then 
Ty^x > 0. Ty^x is clearly explicitly non-symmetric (directional) since it measures the 
degree of dependence of X on Y and not vice versa. 
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B. Effective transfer entropy 



The effective transfer entropy (ETE) was originally introduced by Marchinski et al. in 
Ref. jsj, and it was further substantiated in Refs. 0, El 5|. The ETE, in contrast to the 
STE, accounts for the finite size of a real data set. 

In the previous section, we have defined TY^x{ m J) with the history indices m and I. 
In order to view T Y ^x as a genuine transfer entropy, one should really include in (I3"3"j) the 
whole history of Y and X up to time t m (i.e., all historical data that may be responsible 
for cross-correlations with X tm+1 ). The history is finite only if X or/and Y processes are 
Markovian. In particular, if X is a Markov process of order m+1 and and Y is of order I, 
them TY^x{ m J) is a true transfer entropy. Unfortunately most dynamical systems cannot 
be mapped to Markovian processes with finite-time memory. For such systems one should 
take limits m — > oo and I — > oo. In practice, however, the finite size of any real data set 
hinders this limiting procedure. In order to avoid unwanted finite-size effects, Marchinski 
proposed the quantity 

Tf_+ X (m,l) = T Y ->x(m,l) - Ty schufflcd ^ x (m, I) , (36) 



where K; Clmffled indicates the data shuffling via the surrogate data technique [45j . The surro- 
gate data sequence has the same mean, the same variance, the same autocorrelation function, 
and therefore the same power spectrum as the original sequence, but (nonlinear) phase rela- 
tions are destroyed. In effect, all the potential correlations between time series X and Y are 
removed, which means that Ty schuffled _ s .x("i, I) should be zero. In practice, this shows itself 
not to be the case, despite the fact at there is no obvious structure in the data. The non-zero 
value of Ty schufflcd ^x("7., must then be a byproduct of the finite data set. Definition (1361) 
then ensures that spurious effects caused by finite m and I are removed. 

IV. RENYIAN TRANSFER ENTROPIES 

There are various ways in which one can sensibly define a transfer entropy with Renyi's 
information measure S q . The most natural definition is the one based on a g-analog of 
Eqs. (I32|l-(j33|l ; i.e., 



T$^x(m, i) = s( R \x tm+1 \x tl n-nij- s^ R \x tm+1 \x tl n • ■ ■ n x tm n Y tl n - ■ ■ n Y tl ) 
= /f \x tm+1 ;Y tl n • • • n Y tl \x h n • • • n x tm ) . (37) 

With the help of (1251) and (1301) this can be written in an explicit form as 

T S^x( m J) 

1 , T,Qq(xt 1 ,---,x tm )p q {x tm+1 \xt 1 ,...,x tm ) 
log 2 



1 ~ <? Qq( X t!, ■ ■ ■ , XtmlUtrrt-l+l-, ■ ■ ■ > Htm )P ? ( X t m + 1 \ X tll • • • ) X tm- l Ut m -l + ll • • • J Ut m , 



i , Y,eq( x ti,---i x t m )p q (yt m _ l+1 ,---,yt m \x tl ,...,x tm ) 

io S2 v- - J- — r~7r, : — n 1 — \ • t 6i >) 



1 -Q E Q q ( x ti > • ■ ■ , x t m+1 )p q (yt m . l+1 , ■ ■ ■ , ytm I x h , • • • , x t m+1 ) 
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Here, g q is the escort distribution (fT5|) . One can again easily check that in the limit q — > 1 
we regain the Shanonnian transfer entropy 

The representation (1381) deserves a few comments. First, when the history of Y up to 
time t m has no influence on the next-time-tick value of X (i.e., on X tm+1 ), then from the 
first line in (138]) it follows that T^^ x (m, I) = 0, which indicates that no information flows 
from Y to X, as should be expected. In addition, T^_^ x as defined by (1371) and (1381) takes 
into account only the effect of time series Y (up to time t m ), while the compound effect of 
the time series X (up to time t m ) is subtracted (though indirectly present via correlations 
that exist between time series X and Y). In the spirit of Section Til Bl one may interpret the 
transfer entropy T^y_^ x as a rating factor which quantifies a gain/loss in the risk concerning 
the behavior of X at the future time t m+ \ after we take into account the historical values of 
a time series Y until t m . 

( Ft) 

Unlike in Shannon's case, T^.y^ x = does not imply independence of the X and Y 
processes. This is because T^_^ x (m, I) can also be negative on account of nonlinear pricing. 

(Ft) 

Negativity of T^.y^ x then simply means that the knowledge of historical values of both X 
and Y broadens the tail part of the anticipated PDF for the price value X tm+1 more than 
historical values of X only would do. In other words, an extra knowledge of historical values 
of Y reveals a greater risk in the next time step of X than one would anticipate by knowing 
merely the historical data of X alone. 

Note that, with our definition ( 1371) . Tg.y^x is again explicitly directional since it measures 
the degree of dependence of X on Y and not the other way around, though in this case we 
should indicate by an arrow whether the original risk rate about X tm+l was increased or 
reduced by observing the historical values of Y. 

At this stage, one may introduce the effective Renyi transfer entropy (ERTE) by following 
the same logic as in the Shannonian case. In particular, one can again use the surrogate 
data technique to define the ERTE as 

T^kmJ) = T^ x (m,l) - T^ schua ^ x (m,l). (39) 

Similarly to the RTE, T^^ x (m, I) also accentuates for q G (0, 1) the flow of information that 
exists between the tail parts of distributions; i.e., it describes how marginal events in the time 
series Y influence marginal events in the time series X. Since most of historical data belong 
to the central parts of distributions (typically with well-behaved Gaussian increments), one 
can reasonably expect that for q G (0, 1) the transfer entropy T^^ x (m, I) = T^_^ x (m, I), 
and the surrogate data technique is not needed. This fact is indeed confirmed in our data 
analysis presented in the following section. 



V. PRESENTATION OF THE ANALYZED DATA 



In the subsequent analysis, we use two types of data set to illustrate the utility of Renyi's 
transfer entropy. The first data set consists of 11 stock exchange indices, sampled at a daily 
(end of trading day) rate. The data set was obtained from Yahoo financial portal (historical 
data) with help of the R-code program 46j for the period of time between 2 January 1998 
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and 31 December 2009. These data will be used to demonstrate quantitatively the statistical 
coherence of all the mentioned indices in the form of heat maps and net flows. 

Because we also wish to illustrate our approach quantitatively, we use as a second data set 
time series of 183.308 simultaneously recorded data points from two market indices, namely 
from the DAX index and the S&P500 insex, gathered on a minute-tick basis in the period 
from 2 April 2008 to 11 September 2009. In our analysis, we use complete records, i.e., 
minute data where only valid values for both the DAX index and the S&P500 index, are 
admitted: periods without trading activity (weekends, nighttime, holidays) in one or both 
stock exchanges were excluded. This procedure has the obvious disadvantage that records 
substantially separated in a real time may become close neighbors in the newly defined 
time series. Fortunately, relatively the small number of such "critical" points compared 
to the regular ones prevents a statistically significant error. In addition, due to computer 
data massification one may reasonably expect that the trading activity responds almost 
immediately to external stimuli. For this reason we have synchronized the data series to an 
identical reference time, a master clock, which we take to be Central European Time (CET). 



A. Numerical calculation of transfer entropies 

In order to find the PDF involved in definitions (I34jl (respectively, (136jl ) and ( 1381) (respec- 
tively, ( 1391) ) we use the relative-frequency estimate. For this purpose we divide the amplitude 
(i.e., stock index) axis into N discrete amplitude bins and assign to every bin a sample data 
value. The number of data points per bin divided by the length of the time series then 
constitutes the relative frequency which represents the underlying empirical distribution. In 
order to implement the R-code in the ETE and ERTE calculations we partition the data 
into disjoint equidistant time intervals (blocks), which serve as a coarse-graining grid. The 
number of data points we employ in our calculations is constant in each block. In each block 
only the arithmetic mean price is considered in block-dependent computations. 

It is clear that the actual calculations depend on the number of bins chosen (this is also 
known as the alphabet length). In Ref. it was argued that, in large data sets such as our 
time series, the use of alphabets with more than a few symbols is not compatible with the 
amount of data at one's disposal. In order to make a connection with existing results (see 
Refs. 0,0]), we conduct calculations at fixed alphabet length N = 3. 

For a given partition, i.e., fixed I, TY-+x(m, I) is a function of the block length m. The 
parameter m is to be chosen as large as possible in order to find a stable (i.e., large m 
independent) value for Ty^xijn, 0> however, due to the finite size of the real time series X, 
it is required to find a reasonable compromise between unwanted finite sample effects and 
a high value for m. This is achieved by substituting Ty^x{fn,l) with the effective transfer 
entropy. Surrogate data that are needed in definitions of the ETE ( |36l) and the ERTE 



( 139]) are obtained by means of standard R routines 46[. The effective Renyi and Shannon 
transfer entropies themselves are explicitly calculated and visualized with the help of the 
open-source statistical framework R and its related R packages for graphical presentations. 
The calculations themselves are also coded in the R language. 
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B. Analyzing the daily data — 



heat maps vs. net information flows 



The effective transfer entropies T Y S _^ X and T^y^ x are calculated between 11 major stock 
indices (see the list in Appendix A). The results are collected in three tables in Appendix B 
and applied in the constructions of heat maps and net information flows in Figs. ITHTBl In 
particular, Shannon's information flow is employed in Figs. |8] and [9] while Renyi's transfer 
entropy is used in construction of Figs. [TUHT51 The histogram-based heat map in Fig. [7] 
represents the overall run of the 11 aforementioned indices after the filtering procedure. We 
have used the RColorBrewer package 47| from the R statistical environment which employs 
a color-spectrum visualization for asset prices. In this case the color runs from the green, 
for higher prices, to dark purple, for low price values. 

The heat map in Fig. |8] shows discloses that among the 11 selected markets a substantial 
amount of information flows between the Asia-Pacific region (APR) and the US. One can also 
clearly recognize the strong information exchanges between the APR and European markets 
and the subdominant information flow between the US and Europe. There is comparably 
less information flowing among European markets themselves. This can be credited to the 
fact that the internal European market is typically liquid and well equilibrated; similarly, 
a system in thermal equilibrium (far from critical points) has very little information flow 
among various parts. An analogous pattern (safe for the NY index) can also be observed 
among the US markets. In contrast, the markets within the APR mutually exchange a 
relatively large volume of information. This might be attributed to a lower liquidity and 
consequently less balanced internal APR market. 

The heat maps in Figs. [TO] and [12] bring further understanding. Notably, we can see that 
the information flow within APR markets is significantly more imbalanced between wings 
of the asset distributions (larger color fluctuations) than between the corresponding central 
parts. This suggests low liquidity risks. A similar though subordinate imbalance in the 
information transfer can also be observed between the US and APR markets. 

Understandably more revealing are the net information flows presented in Figs. |9| [TTland 
[T3l The net flow F Y ^x is defined as F Y ^ X = T Y ~>x — Tx-^y- This allows one to visualize 
more transparently the disparity between the Y — > X and X — > Y flows. For instance, in 
Fig. [9] we see that substantially more information flows from the APR to the US and Europe 
than vice versa. Figs. [11] and [13] then demonstrate more specifically that the APR — > Europe 
flow is evenly distributed between the central and tail distribution parts. From the net flow 
in Figs. |9] [H] and [13] we can also observe an important yet comparably weaker surplus of 
information flow from Europe towards the US. This interesting fact will be further addressed 
in the following subsection. 

Note also that t[ % Pk500 _, NY , r o.8?sp&50o->jvy and t oS-,ny^dj have negative values. These 
exceptional behaviors can be partly attributed to the fact that both the SP&500 and DJ 
indices are built from indices that are also present in the NY index and hence one might 
expect unusually strong coherence between these indices. From Section IIVI we know that 
negative values of the ERTE imply a higher risk involved in a next-time-step asset-price 
behavior than could be predicted (or expected) without knowing the historical values of the 
source time series. The observed negativity of T^. X ^ Y thus means that when some of the 



18 



ignorance is elevated by observing the time series X a higher risk reveals itself in the nearest- 
future behavior of the asset price Y. Analogously, negativity of T{ 5 . X ^ Y corresponds to a 
risk enhancement of the non-risky (i.e. close-to-peak) part of the underlying PDF. 



C. Minute-price information flows 

Here we analyze the minute-tick historic records of the DAX and S&P500 indices collected 
over the period of 18 months from 2 April 2008 to 11 September 2009. The coarse-grained 
overall run of both indices after the filtering procedure is depicted in the histogram-based 
heat map in Fig. UJ 

Without any a prior knowledge about the Markovian (or non-Markovian) nature of the 
data series, we consider the order of the Markov process for both the DAX and S&P500 
stocks to be identical, i.e., the price memory of both indices is considered to be the same. 
The latter may be viewed as a "maximally unbiased" assumption. At this stage we eliminate 
the surrogate data and consider the RTE alone. The corresponding RTEs for q = 1.5 and 
q = 0.8 as functions of block lengths are shown in Figs. HH and [151 respectively. There we 
can clearly recognize that for m ~ 200 — 300 minutes there are no new correlations between 
the DAX and S&P500 indices. So, the underlying Markov process has order (or memory) 
roughly 200 — 300 minutes. 

The aforementioned result is quite surprising in view of the fact that autocorrelation 
functions of stock market returns typically decay exponentially with a characteristic time of 
the order of minutes (e.g., ~ 4 mins for the S&P500 48, 4{|), so the returns are basically 



uncorrelated random variables. Our result, however, indicates that two markets can be 
intertwined for much longer. This situation is actually not so surprising when we realize 



that empirical analysis of financial data asserts (see, e.g., 50[) that autocorrelation functions 



of higher-order correlations for asset returns have longer decorrelation time which might span 



up to years (e.g., a few months in the case of volatility for the S&P500 [49]) . It is indeed a 
key advantage of our approach that the nonlinear nature of the RTE naturally allows one 
to identify the existing long-time cross- correlations between financial markets. 

In Fig. [161 we depict the empirical dependence of the ERTE on the parameter q. Despite 



the fact that the RE itself is a monotonically decreasing function of q (see, e.g., Ref. [26|) 
this is generally not the case for the ERTE (nor for the conditional RE). Indeed, the ERTE 
represents a difference of two REs with an identical q (see Eq. (|37l)). and as such it may be 
neither monotonic nor decreasing. The functional dependence of the ERTE on q nevertheless 
serves as an important indicator of how quickly REs involved change with q. 

The results reproduced in Fig. [16] quantitatively confirm the expected asymmetry in the 
information flow between the US and European markets. However, since the US contributes 
more than half of the world's trading volume, it could be anticipated that there is a stronger 
information flow from big US markets towards both European and APR markets. Yet, 
despite the strong US trading record, our ERTE approach indicates that the situation is 
not so straightforward when the entropy-based information flow is considered as a measure 
of market cross-correlation. Indeed, from Figs. [91 [11] and [13] we could observe that there 
is a noticeably stronger information flow from the European and APR markets to the U.S. 
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markets than than vice versa. Fig. [16] extends the validity of this observation also to short 
time scales of the order of minutes. In particular, from Fig. [16] we clearly see that flow from 
the DAX to the S&P500 is stronger than the reverse flow. It is also worth of noting that 
this Europe-US flow is positive for all values of q, i.e., for all distribution sectors, with a 
small bias towards tail parts of the underlying distribution. 

VI. CONCLUDING REMARKS 

Transfer entropies have been repeatedly utilized in the quantification of statistical co- 
herence between various time series with prominent applications in financial markets. In 
contrast to previous works in which transfer entropies have been exclusively considered only 
in the context of Shannon's information theory, we have advanced here the notion of Renyi's 
(i.e. non-Shannonian) transfer entropy. The latter is defined in a close analogy with Shan- 
non's case, i.e., as the information flow (in bits) from Y to X ignoring static correlations due 
to the common historical factors such as external agents or forces. However, unlike Shan- 
non's transfer entropy, where the information flow between two (generally cross-correlated) 
stochastic processes takes into account the whole underlying empirical price distribution, 
the RTE describes the information flow only between certain pre-decided parts of two price 
distributions involved. The distribution sectors in question can be chosen when Renyi's 
parameter q is set in accordance with Campbell's pricing theorem. Throughout this paper 
we have demonstrated that the RTE thus defined has many specific properties that are 
desirable for the quantification of an information flow between two interrelated stochastic 
systems. In particular, we have shown that the RTE can serve as an efficient rating factor 
which quantifies a gain or loss in the risk that is inherent in the passage from X tm to X tm+1 
when a new information, namely historical values of a time series Y until time t m , is taken 
into account. This gain/loss is parameterized by a single parameter, the Renyi q parame- 
ter, which serves as a "zooming index" that zooms (or emphasizes) different sectors of the 
underlying empirical PDF. In this way one can scan various sectors of the price distribution 
and analyze associated information flows. In particular, the fact that one may separately 
scrutinize information fluxes between tails or central-peak parts of asset price distributions 
simply by setting q < 1 or q > 1, respectively, can be employed, for example, by finan- 
cial institutions to quickly analyze the global (across-the-border) information flows and use 
them to redistribute their risk. For instance, if an American investor observes that a certain 
market, say the S&P500, is going down and he/she knows that the corresponding NASDAQ 
ERTE for q < 1 is low, then he/she does not need to relocate the portfolio containing re- 
lated assets rapidly, because the influence is in this case slow. Slow portfolio relocation is 
generally preferable, because fast relocations are always burdened with excessive transaction 
costs. Let us stress that this type of conduct could not be deduced from Shannon's transfer 
entropy alone. In fact, the ETE suggests a fast (and thus expensive) portfolio relocation as 
a best strategy (see Figs. I91TTT1 and [T3l) . 

Let us stress that applications of transfer entropies presented quantitatively support the 
observation that more information flows from the Asia-Pacific region towards the US and 
Europe than vice versa, and this holds for transfers between both peak parts and wing parts 
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of asset PDFs; i.e., the US and European markets are more prone to price shakes in the 
Asia-Pacific sector than the other way around. Besides, information-wise the US market 
is more influenced by the European one than in reverse. This interesting observation can 
be further substantiated by our DAX versus S&P500 analysis, in which we have seen that 
the influx of information from Europe is to a large extend due to a tail-part transfer. The 
peak-part transfer is less pronounced. So, although the US contributes more than half of 
the world's trading volume, our results indicate that this is not so with information flow. 
In fact, the US markets seem to be prone to reflect a marginal (i.e., risky) behavior in both 
European and APS markets. Such a fragility does not seem to be reciprocated. This point 
definitely deserves further closer analysis. 

Finally, one might be interested in how the RTE presented here compares with other 
correlation tests. The usual correlation tests take into account either the lower-order corre- 
lations (e.g., time- lagged cross-correlation test and Arnhold et al. interdependence test) or 
they try to address the causation issue between bivariate time series (e.g., Granger causality 
test or Hacker and Hatemi-J causality test). Since the RTE allows one to compare only 
certain parts of the underlying distributions it also works implicitly with high-order cor- 
relations, and for the same reason it cannot affirmatively answer the causation issue. In 
many respects such correlation tests bring complementary information with respect to the 
RTE approach. More detailed discussion concerning multivariate time series and related 
correlation tests will be presented elsewhere. 

Acknowledgments 

This work was partially supported by the Ministry of Education of the Czech Republic 
(research plan MSM 6840770039), and by the Deutsche Forschungsgemeinschaft under grant 
K1256/47. 



21 



Appendix A 



In this appendix we provide a brief glossary of the indices used in the main text. The no- 
tation presented here conforms with the notation typically listed in various on-line financial 
portals (e.g., Yahoo financial portal). 



Indices 


Description 


Country 


GSPC 


Standard and Poor 500 (500 stocks actively traded in the U.S.) 


USA 


GDAXI 


Dax Indices (stock of 30 major German companies) 


Germany 


ATX 


The Austrian Traded Index is the most important stock market index 
of the Wiener Borse. The ATX is a price index and currently consists 
of 20 stocks. 


Austria 


SSMI 


The Swiss Market Index is a capitalization-weighted index of the 20 
largest and most liquid stocks. It represents about 85% of the free-float 
market capitalization of the Swiss equity market. 


Swiss 


AORD 


All Ordinaries represents the 500 largest companies in the Australian 
equities market. Index constituents are drawn from eligible companies 
listed on the Australian Stock Exchange 


Australia 


BSESN 


The BSE Sensex is a market capitalized index that tracks 30 stocks from 
the Bombay Stock Exchange. It is the second largest exchange of India 
in terms of volume and first in terms of shares listed. 


India 


HSI 


The Hang Seng Index denoted in Hong Kong stock market. It is used to 
record and monitor daily changes of the largest companies of the Hong 
Kong stock market. It consist of 45 Companies. 


Hong Kong 


N225 


Nikkei 225 is a stock market index for the Tokyo Stock Exchange. It 
is a price- weighted average (the unit is yen), and the components are 
reviewed once a year. Currently, the Nikkei is the most widely quoted av- 
erage of Japanese equities, similar to the Dow Jones Industrial Average. 


Japan 


DJA 


The Dow Jones Industrial Average also referred to as the Industrial 
Average, the Dow Jones, the Dow 30, or simply as the Dow; is one of 
several U.S. stock market indices. First published in 1887. 


USA 


NY 


iShares NYSE 100 Index is an exchange trading fund, which is a security 
that tracks a basket of assets, but trades like a stock. NY tracks the SE 
U.S. 100; this equity index measures the performance of the largest 100 
companies listed on the New York Stock Exchange (NYSE). 


USA 


IXIC 


The Nasdaq Composite is a stock market index of all of the common 
stocks and similar securities (e.g., ADRs, tracking stocks, limited part- 
nership interests) listed on the NASDAQ stock market, it has over 3.000 
components. 


USA 
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Appendix B 



In this appendix we specify explicit values of effective transfer entropies that are employed 
in Section [V] These are calculated for alphabet with N = 3. 

GSPC GDAX1 ATX SSMI AORD BSESM HSI N225 DJA NY (XIC 



GDAXI 


0,00000 0,64294 0,20563 
0,77440 0,00000 0,12875 


0,58809 
1,06341 


0,13525 0,13135 
0,23264 0,07227 


0,24831 
0,39049 


0,12437 0,18488 0,31810 
0,06604 0,62008 0,71447 


0,63655 
0,95265 




0,30296 0,17621 0,00000 


0,20245 


0,25843 0,31853 


0,11564 


0,08610 0,20097 0,29119 


0,22579 




0,53800 0,69405 0,13986 
1,36056 1,51079 0,65343 


0,00000 
1,86269 


0,25384 0,14016 
0,00000 0,64022 


0,40735 
1,46986 


0,05545 0,54607 0,49822 
0,25226 1,70907 1,55052 


0,71236 
1 ,89641 




0,06470 0,06102 0,10300 


0,15650 


0,12057 0,00000 


0,18340 


0,03929 0,13135 0,07823 


0,29994 




0,32333 0,52722 0,21966 


0,65140 


0,28436 0,37382 


0,00000 


0,15690 0,39511 0,39076 


0,79652 




0,96362 0,99773 0,65630 


1,51051 


0,84226 1,32050 


0,83978 


0,00000 1,32331 1,24970 


1 ,06653 




0,42696 0,77354 0,15381 
1,54040 0,56323 0,15345 


0,95252 
0,57273 


0,18801 0,14302 
0,21266 0,11554 


0,38341 
0,22785 


0,08907 0,00000 1,75167 
0,10262 0,22374 0,00000 


1,10667 
0.80568 




| 0,13514 0,23602 0,04650 


0,24937 


0,08140 0,06606 


0,17859 


0,04118 0,17067 0,18350 


0,00000 



FIG. 4: Numerical data for the ETE that are used to generate Figs. [8] and O 





GSPC GDAXI 

0,00000 0,75542 


ssam 

0,23044 


SSMI 

0,68704 


AORD 

0,18199 


BSESN HSI N225 DJA 

0,14466 0,29378 0,13147 0,20075 


NY IXIC 

0,29827 0,70910 




0,87416 0,00000 
0,29124 0,17517 
0,66533 0,83485 


0,16289 
0,00000 
0,14882 


1,26024 

0,20228 
0,00000 


0,29135 
0.25199 
0,30681 


0,06898 0,46002 0,06911 0,74582 
0,31975 0,10526 0,06322 0,20047 
0,16143 0,48324 0,06393 0,68161 


0,80991 1,06154 
0.26042 0,21941 
0,61874 0,88317 




1,43285 1,63308 
0,08182 0,08539 


0,68199 
0,09824 


2,00807 
0,17994 


0,00000 
0.13056 


0,88969 1,56056 0,23869 1,75402 
0,00000 0,22778 0,03927 0,14104 


1,65517 1,96782 
0,07924 0,33590 




0,39850 0,60735 


0,20366 


0,78253 


0,33898 


0,45236 0,00000 0,15589 0,48579 


0,47254 1,01986 




0,99591 1,00682 


0,87902 


1,50917 


0,85318 


1,33146 0,83274 0,00000 1,33389 


1,25635 1,10432 




0,45633 0,85978 

■ i 51' 0,69553 


0,16292 
0,17910 


1,10406 
0,66445 


0,24172 
0.25290 


0,15696 0,44653 0,09256 0,00000 
0,13395 0,26325 0,10823 0,26325 


1,11094 1,15357 
0,00000 0,83430 


FIG. 5: 


0,15980 0,31494 

: Numerical dat 


0,05382 0,34622 0,12063 
a for the ERTE that £ 


0,09491 0,27096 0,04925 0,22103 0,21860 0,00000 
ire used to generate Figs. fTOlandfTT] q = 1.5. 


0,00000 0,59985 


0,19689 


0,55001 


0,11760 


0,12640 0,23150 0,12188 0,18844 


0,35837 0,59674 




0,73595 0,00000 
0,30781 0,17697 


0,11610 
0,00000 


0,99500 
0,20278 


0,21072 
0,26107 


0,07367 0,36417 0,06500 0,57206 
0,31832 0,11974 0,08729 0,20142 


0,67796 0,91007 
0,29575 0,22852 




0,48954 0,64025 


0,13663 


0,00000 


0,23401 


0,13220 0,37856 0,05228 0,49472 


0,45253 0,64689 




1,33377 1,46457 


0,64249 


1,80720 


0,00000 


0,82122 1,43513 0,25782 1,69300 


1,51069 1,87012 




0,08593 0,07945 


0,11216 


0,14773 


0,11678 


0,00000 0,16665 0,03931 0,13135 


0,07305 0,28653 




0,29513 0,49715 


0,22674 


0,60207 


0,26371 


0,34477 0,00000 0,15772 0,36080 


0,35985 0,71247 




0,98648 0,99462 


0,84311 


1,51145 


0,83787 


1,31649 0,84356 0,00000 1,31936 


1,24723 1,07973 




0,44051 0,74738 


0,15077 


0,89469 


0,16760 


0,13769 0,35985 0,08787 0,00000 


-2,57465 1,08989 




-1,98440 0,54747 


0,15095 


0,53761 


0,19733 


0,10858 0,21460 0,10079 0,21396 


0,00000 0,80749 




0,12602 0,20599 


0,04414 


0,21223 


0,06682 


0,05549 0,14385 0,03314 0,14862 


0,17055 0,00000 



FIG. 6: Numerical data for the ERTE that are used to generate Figs. [T2]and[T3l q = 0.8. 
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FIG. 7: Histogram-based (i.e., non-entropic) heat map between the 11 stock indices listed in 
Appendix A. 
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GSPC GDAXI ATX SSMI AORD BSESN HSI N225 DJA 



FIG. 8: Heat map of Shannon's effective entropy between the 11 stock indices listed in Appendix A. 
Alphabet size N = 3. 



27 



N=3 



IXIC 



NY 



DJA 



N225 



HSI 



BSESN 



AORD 



SSMI 



ATX 



GDAXI 



GSPC 



mm 



a a a a am 



A A 



V 

ft 



4wl 



to 0.5 -t> 
0.5 to 1 

1 to 1.5 

1.5 to 2 ♦ 



FIG. 9: Net flow Fy^x of effective Shannon transfer entropies between the 11 stock indices listed 
in Appendix A. Alphabet size A?" = 3. 
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FIG. 10: Heat map of Renyi's effective entropy between the 11 stock indices listed in Appendix 
q = 1.5. Alphabet size N = 3. 
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N=3, q=1.5 
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FIG. 11: Net flow Fy^x of the effective Renyi transfer entropies listed in Appendix A; q = 1.5. 
Alphabet size = 3. 
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GSPC GDAXI ATX SSMI AORD BSESN HSI N225 DJA NY IXIC 



FIG. 12: Heat map of Renyi's effective entropy between the 11 stock indices listed in Appendix A; 
q = 0.8. Alphabet size N = 3. 
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N=3, q=0.8 
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FIG. 13: Net flow Fy-^x of effective Renyi transfer entropies listed in Appendix A; q = 0.8. 
Alphabet size N = 3. 
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FIG. 14: Effective Renyi transfer entropy 5 SP ^ 5Q0 ^ DAX (m,m) for a 3- letter alphabet as a 
function of the block length m. DAX and SP&500 minute prices are employed. The correlation 
time is between 200 — 300 minutes. 
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FIG. 15: Effective Renyi transfer entropy Tq S SP ^ 5Q0 ^ DAX (m,m) for a 3- letter alphabet as a 
function of the block length m. DAX and SP&500 minute prices are employed. The correlation 
time is between 200 — 300 minutes. 
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FIG. 16: The ERTE as a function of q. The alphabet size is set to N = 3. DAX and SP&500 
minute prices are employed. The red curve corresponds to T^gp S gj. 50Q ^ DAX (m, m) while the green 
curve denotes T$^^ SPbsoo (m,m). 
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